Towards Semantic Role Labeling of Hand-drawn Sketches

In this paper we propose an exploratory study for the application of Computational Linguistics techniques such as the Semantic Role Labeling of natural languages (SRL) to the case of sketch understanding. Our approach originated from a preliminary psychological experiment in which participants were asked to produce hand-drawn sketches from sentences embedding various semantic features; the effectiveness of so-obtained sketches was then assessed by other participants who were asked to match the sketches with the original sentences. This paper will present a preliminary study for automating the extension of Semantic Role Labeling to the case of hand-drawn sketches and will provide an initial example of its applications consisting in the evaluation of the drawing style consistency of sketches. This study will seek to enrich the body of research related to semantic sketch understanding.